posted 06-21-2007 08:31 AM         

Thanks. What I'm trying to get at with my question is whether there's a consensus on the relative diagnostic value (or lack thereof) of recognizing pneumo patterns in any given form , as opposed to simply using auto-measuring s/w calipers.

Clearly, using caliper measurements on, say, an LX4000 is much easier than interpreting the pneumo patterns, but is relying the RLL alone always the wiser choice?

I look forward to others' views on this subject.

posted 06-21-2007 08:50 AM         

I use a sub-scoring criteria for the pneumos----as do many others. If one spot has suppression with goofy I and E, that gets a 2 point subscore---and if the comparative spot has only suppression, than that spot gets 1 point. The 2 pointer wins, and gets the plus or minus score on my score sheet. I don't use any measuring devices as I have been cautioned by senior examiners to not seperate fly $h!+ from pepper. I do however respect those that use the templates and computer tools and do not assert that they are of no use. It's just not my manner of scoring. Good post Dan

posted 06-21-2007 09:21 AM            

Learned that one rock climbing, and using copperhead protection placements in small cracks. Place em, pound em, paste em, x em, and sniff em. The last thing you do before putting your body weight on a placement, when you'r dangling far above the terra firma, is to sniff it. If it stinks, then don't use it. There's not much that's less fun than zippering out a whole string of pieces just because one fails. But its a really good way to scare the bejeezus out of block-headed gang-bangers who act like there afraid of nothin.

Sniff test is really just a gut check for abnormal looking data: Irregular patterns, rates converging, diverging upper and lower pneumos, and the presence of distortion artifacts, and changes in respiratory functioning that suggest something other than authentic autonomic activity.

The pneumos may contain the richest information available to us. With the greatest volume of phenomena that influence how the data develop: conscious attention, relaxation, CMs, health (allergies, illness, medicines), autonomic/sympathetic activity, temperature, and behavioral/perhipheral activity, and probably more. Just look at the long list of pneumograph features in some of the arcane scoring systems, and you'll see that pneumos include more possible tracing features than any other component.

The downside of all that is that same range and volume of data causes the pneumos to be the most potentially unreliable data (unreliable meaning that different examiners attend to different atures).

I know you said leave the algorithms out of it, but OSS was originally intended for handscoring - so here goes. The OSS includes a procedure that may serve to protect against unreliable pneumo data - zeroing the data when the upper and lower pneumo produce values of opposite numerical sign. Its quite possible that opposite signs (between upper and lower pneumos) are more likely to occur with data of abnormal quality (e.g., autonomic data are adultered by attention, behavior, illness, CMs, or what-have-you), while normal autonomic activity may be more likely to produce upper and lower numerical signs of uniform value. This is just my working hypothesis.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-21-2007 12:33 PM         

Dan, to address your original question, I do not ‘measure’ RLL, but rather ‘eyeball’ it over all for the greater ‘suppression’; I went the route of purchasing a quality measurement wheel and ‘measured’ the RLL on a few exams but quickly became frustrated- due to my ‘measure and re-measuring’ because of inadvertant backward movement on the wheel while tracing, or my hand tremors adding a few millimeters, or traversing slightly outside the pneumo trace line. Then began to question just ‘what’ to measure, - inhalation only (tidal volume) or both inhalation and exhalation, and for ‘how long’- then came DACA with ‘start times’ for measurement not necessarily being at question onset. Then along came ‘calipers’ in manufacturer’s software which made things much easier to “measure”- but more often than not “what it measured” and “what I saw” seemed so damn different, not mention a lot of time consuming ‘note taking’ to figure ratio’s and the like- only to ‘weigh in’ the baseline shift when there was no significant difference in ratio’s.

Personnally, I do not necessarily go by “pattern recogonition” so much either. Rather I generally ‘eyeball’ and mentally evaluate\analyse ‘why’ and ‘what may be the cause’ for a change in inspiratory time; expiratory time; I/E ratio; tidal volume or baseline shifts had occurred- adding in which has the greater appearance of supression; and then assigning a +/- 1 score. I evaluate both thoracic and abdominal- and then ‘combine’ (not add) the scores. Rare for me is a 2, and rarer still is a 3. Rnelson’s viewpont and mine probably are not too far apart.

Frankly- I tend to think a ‘third’ pneumo channel could be of interest, the ‘third’ pneumo channel consisting of combing both the upper and lower pneumo tubes to input into a DAS channel to produce one additional trace line to reflect ‘total tidal volume changes’ for the subject. Then assign a +/-1 only to each of the three trace lines. By combining both the upper and lower pneumo’s to produce one trace line- it just seems to me computerized RLL measurements ‘for supression’ would be more representative.

There’s been some pretty good topics here lately- just have to find the time to type.

posted 06-21-2007 01:06 PM            

You are correct, of course, that RLL collapses the myriad of features to a single index that can be reliably and mechanically measured.

Timm always gets the cite on this, but he credits Horvath for the suggestion.

While RLL is robust and reliable, it is probably not immune to distortion artifacts, and therefore not without some requirements about data quality.

We know that respiratory activity is normally autonomic (with greater activity during periods of effort, such as hauling our happy carcasses up some steep slope), though easily brought under the voluntary control of the peripheral nervous system.

We do not know whether RLL data and our present scoring/interpretation rule is robust with autonomic data that is adultered or artifacted with peripheral nervous system behavior (or perhaps some of that fine shaking that results from nebulizer/inhaler meds for things like asthmatic bronchitis)

I can think of several recent exams in which we observe a distorion artifact in the form of a prominent downward tracing segment and return to normal around the point of answer. Those segments serve to lengthen the RLL and possibly distort the ratios and scores.

So, it is important to know/remember how the method was trained.

Kircher and Raskin (1988) described smothing the data for several 1/2 second cycles around the point of answer in the pneumos. So, it seems RLL ratios may not be intended to be robust against those answering artifacts.

Stoelting presently has some really cool features for smoothing and interpolating distorted tracing segments.

I suspect that our general understanding of RLL is non-robust against non-normal data.

Remember: garbage in = garbage out.

Gotta go, have a date with some altititude.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-21-2007 07:14 PM            

Ben

posted 06-21-2007 08:38 PM         

I'd be grateful if someone who manually calculates RLL to please post the formula for arriving at the ratios, and identify the thresholds for noticeable/significant/dramatic differences.

Also, should the calipers always be placed at the point of stimulus onset? Sometimes it appears (to my naked eye, at least) that answering distortions might have an effect on line length... Does anyone place the calipers at the point of answer?

posted 06-22-2007 08:51 AM            

The 10 second time span came from research done by Dr. Kircher, and you start at stimulus onset. Answers will cause distortions, and OSS didn't consider them - that I recall anyhow. CPS's scoring algorithm, which also uses 10 seconds, does do something to account for the answer distortions to questions. I think I recall Dr. Kircher objecting to Polyscore because it didn't do that, among other things.

DACA now looks at the entire reaction - wherever it starts (as long as it's timely), so they could begin considering RLL at the answer, yes. Then, for example, they could go out 15 seconds (or wherever homeostasis returns). They then use that time period (in this example, 15 seconds) to determine the time period for the question to which it's being compared. They would score 15 seconds of RLL wherever the reaction started - even if it started at stimulus onset in that question - and they'd go out 15 seconds, even if they are then measuring relief.

It's a little complicated to explain here, and it will be in the AAPP handbook, due out this month, and, for you Dan, I'll have a slide-show during the upcoming training.

posted 06-22-2007 01:55 PM            

quote:

---

Do an experiment. Take 10-20 previous tests and subtract all pneumo scores from the final totals----and then calculate the affects. Do 1/3 of the historic calls become opposing calls (on the 3 point scale)? What's the rate of inconclusives-----have you used the pneumos as a sort of loaning institute to make a firm call when without them no such affirmation existed? Your "art debate" is probably more apparent than ever-----and I've agreed with much of what you've written in the past (my agreement might be unwelcome as my sheep are all black.)

---

Once again this isn't exactly hand scoring, but OSS, the associated ratios, and the use of RLL is a hand scoring paradigm, so here goes again.

Using the OSS training sample of N=292 ZCT cases, I've scored the data with and without the pneumograph data, using the OSS-3/Senter algorithm that has been under development since about August or November last year.

Raw figures are these

% correct w pnueumos = 93.9
% correct wo pneumos = 88.8

% inc w pneumos = 4.5
% inc wo pneumos = 5.5

Its really only a couple of percentage points, but it doesn't look all that good. Coupled with the probably meaningless change in inconclusive rates

Now, just because we see a difference, doesn't mean that difference is meaningful. For that we need a signficance test - which is quite inconvenient (and requires advanced statistics) when we are talking about binary result data.

However, I just happen to have all the tools to perform that task, on this fast new computer.

So, I'm completing a double-bootstrap t-test to determine the significance of that difference.

The double bootstrap is a resampling procedures that provides estimated values (mean and variance) necessary to complete the t-test. First a single bootstrap resample of the M=292 sets of data are constructed from the N=292 cases. We use the mean of those M=292 samples of N=2929 to get our mean estimate. Then for each of those M=292 resample sets, we select another 292 resample sets of N=292 (that's 292 sets of 292 x 292)to achieve an estimate of the variance that we would around our mean estimates if we had the opportunity to construct and work with numerous (lots and lots and lots) separate samples of the population (which we don't and which is why we use advanced statistics like bootstrapping). Yep, it takes about 24,000,000 random selections to test the significance in this way, but it does provide rather robust results.

How much you wanna bet the difference in accuracy is significant while difference in inconclusives is not?

------

Done.

% Correct T=3.0404 p=<.001 (significant)

% INC T=-.8539 p=.197 (not significant)

Its probably too soon to get rid of the pneumos completely.

Being wrong about an assumption is no problem. Not admitting it is a big problems – and probably causes as much trouble in this world as any form of evil.

In science, our job is usually to try to prove our assumptions wrong (not correct). Then we go back to the drawing board and make some more assumption and try to prove those wrong too. Scientists who aren't willing to prove themselves or be proved wrong aren't scientists - they are apologists who have formulated their conclusions (not questions) before their experiments, and then set out to justify those conclusions (sometimes in spite of the data).

This does not suggest that pneumo data are great data. They are inherently noisy and unreliable.

I don't think I'd suggest not scoring pneumos, but I would continue to advocate zeroing or not scoring bad data. By bad data I mean abnormal looking data, that is suspiciously different than normal looking autonomic respiration data: substantial changes in rate, amplitude, gross answering artifacts, converging or diverging thoracic and abdominal data, movement, and other forms of distortion, including coughing, sniffing, sneazing, all the stuff-head-fever-so-you-can-rest stuff that pneumograph data are non-robust against.

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-22-2007 05:47 PM            

(Use only in a well-ventilated polygraph lab)

--------

I should have indicated earlier that the data above don't really contradict your earlier point.

If you find that 1/3 of your calls change after removing pneumo scores, all that really suggests is that some of what we were taught in polygraph school, regarding scoring pneumos, may need to be revisited.

It seems to illustrate the difference between RLL and traditional pneumo scoring.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)